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Preface 


In 2011, | published a report titled The Common Core State Standards Initiative: 
an Overview. This report evaluated decision-making in the Common Core State 
Standards Initiative as the change process moved from research, development 
and diffusion activities to adoption of the standards by the states. Interest in 
comparing curriculum trends in the USA and Australia led to the development of 
a report titled States’ Implementation of the Common Core State Standards and 
the Australian Curriculum: a Comparison of the Change Process in Two 
Countries. Published in June 2015, this report represented an attempt to apply 
a model of the implementation process based on a delivery framework to gauge 
the strength of each state’s capacity to implement the Common Core State 
Standards or Phase One of the Australian Curriculum. 


Late in 2015, | initiated a study to provide a rationale for developing a delivery 
plan to build the capacity of schools in selecting, procuring and using 
instructional materials that are aligned to the Australian Curriculum. This report, 
Aligning curriculum materials with the Australian Curriculum: What is happening 
in the field and what needs to be done?, which was published in February 2016, 
recommended establishing an entity similar to EdReports.org. 


Lauren Weisskirk, EdReports Chief Strategy Officer, contributed to the 
development of this report by reviewing the draft and offering comments. This 
collaboration led to a decision in June 2016 to develop an article based on a 
review of research studies investigating the role of materials and an 
investigation of initiatives launched in the USA to evaluate materials. Focusing 
on decision-making involved in the change process for planning and structuring 
EdReports.org, the article, The Common Core State Standards and the Role of 
Instructional Materials: a Case Study on EdReports.org., was published in 
volume 8, number 1-2 of the International Association for Research on 
Textbooks and Educational Media’s e-Journal. Publication of the article led 
Lauren Weisskirk to commission a short article, Familiarising Australian 
policymakers and educators with EdReports.org, which was published on 
EdReports.org’s website in March 2017. 


Dissemination of the report and the article to Senator Simon Birmingham, the 
Minister for Education and Training, in November 2016 led to consideration of 
the recommendation by officials of the Australian Government Department of 
Education and Training. During a meeting held at Canberra in September 2017, 
the Australian Government Department of Education and Training’s Director of 
Curriculum reported having discussed this concept with representatives from 
the Australian Curriculum, Assessment and Reporting Authority and Education 
Services Australia. Officials from the three organisations recognised that it 
would be timely to proceed with this initiative, since the Australian Curriculum 
was close to being fully implemented across Australia. An outcome of the 
meeting was an agreement to develop a proposal for submission to the 
Education Council. A proposal, however, did not eventuate following the 
meeting due to the officials’ work commitments and subsequent staff changes. 


Early in 2017, | became interested in revisiting the topic of the implementation 
of the Australian Curriculum to identify the impact of Phases Two and Three. 
This report, A Comparison of the Change Process in States’ and Territories’ 


Implementation of the Australian Curriculum, which was published in March 
2018, applied the same model of implementation based on a delivery 
framework to gauge the strength of states’ and territories’ capacities to 
implement the Australian Curriculum. 


This report was disseminated to the federal, state and territory ministers for 
education. South Australia’s Minister for Education and Chair of the Education 
Council, John Gardner referred the report to Rick Persse, Chief Executive of the 
South Australia Department of Education. A meeting was arranged with the 
Department of Education’s Executive Director for Strategic Policy and External 
Relations. During the meeting held at Adelaide in August 2018, it was 
suggested that further communications with officials from the Australian 
Government Department of Education and Training and the Australian 
Curriculum, Assessment and Reporting Authority should be initiated about 
developing a proposal for submission to the Education Council, once the new 
National School Reform Agreement had been released. 


In March 2018, the Australian Government released the report, Through Growth 
to Achievement: Report of the Review to Achieve Educational Excellence. 
Following release of the report, the federal, state and territory governments 
engaged in consultations during 2018 to develop the National School Reform 
Agreement setting out eight reforms to be implemented from 2019 to 2023. 
Each state and territory established a bilateral agreement with the Australian 
Government setting out state-specific reform actions. 


In communications with Lauren Weisskirk early in 2018, | suggested developing 
another article to review new research studies investigating the role of materials 
and to evaluate implementation of EdReports program. | wish to acknowledge 
the contributions made by the following people with regard to particular aspects 
relating to this article. Lauren Weisskirk and Mark LaVenia, EdReports Data 
Strategist, are thanked for identifying relevant reports and policy papers to 
review. Mark LaVenia also referred me to relevant documents about EdReports 
activities, answered queries about EdReports operations, referred specific 
issues to EdReports Executive Director, Eric Hirsch, for response, and reviewed 
the draft for the article. Morgan Polikoff, Associate Professor of Education at the 
University of Southern California, provided publications and documents relevant 
to a study on the effects of textbook adoptions in California, Florida, Illinois, 
New York and Texas as well as reviewing the draft for the article. Nancy 
Rodriguez, Press Secretary Communications for the Council of Chief State 
School Officers provided information about the Instructional Materials 
Professional Development Network. 


Biographical Note 


Michael Watt taught in several secondary schools in Tasmania, and worked as 
an education officer in the Tasmania Department of Education. He holds 
masters’ degrees in educational studies and education from the University of 
Tasmania, and a doctorate in education from the University of Canberra. He 
currently works as an education consultant. 


Abstract 


The purpose of this study was to analyse the findings of research studies 
investigating the role of instructional materials, review issues and 
recommendations referring to instructional materials in policy papers, and 
evaluate the decision-making process in relation to the development, 
diffusion and adoption of EdReports program. Content analysis was used to 
analyse the subject matter of reports and policy papers. A decision-oriented 
evaluation model was used to analyse planning, structuring, implementing 
and recycling decisions occurring in the change process’ within 
EdReports.org. The results showed that the wave of research investigating 
the role of materials has continued with the publication of new reports, the 
release of policy papers, initiation of a study investigating adoption patterns 
and commencement of new projects by EdReports.org. The findings showed 
that researchers are investigating the effects of decision-making on the 
selection of high-quality, standards-aligned materials, foundations and non- 
profit organisations are providing recommendations to policymakers to link 
high-quality, standards-aligned materials with professional learning, and 
EdReports.org has developed, diffused and adopted a program to provide 
information about high-quality, standards-aligned materials to the education 
community. 


Introduction 


In a previous volume of this journal, Watt (2016) reported on the role that 
instructional materials play in American education within the context of the 
Common Core State Standards (CCSS). In 2009 and 2010, the NGA Center for 
Best Practices and the Council of Chief State School Officers (CCSSO) 
oversaw the development of the CCSS by content experts in English Language 
Arts and Mathematics. Released in June 2010, the CCSS were adopted by 46 
states. Opposition from conservative, anti-Common Core groups, which peaked 
in 2014, led 11 states to revise the CCSS to a major extent. From 2011 to 
2013, Achieve oversaw content experts use A Framework for K-12 Science 
Education, developed by the National Research Council of the National 
Academy of Sciences, to develop the Next Generation Science Standards 
(NGSS). Released in April 2013, the NGSS had been adopted by 19 states up 
to November 2017. 


Implementation of the CCSS prompted researchers to investigate the critical 
role of materials in the implementation process and policymakers to introduce 
several initiatives to evaluate the quality of lessons and units, print-based and 
digital materials and their alignment to the CCSS. In the previous article, Watt 
synthesised the findings of reports on four research studies investigating the 
role of materials and focused on the background, current status and 
interactions between actors to produce a case study on EdReports.org. 


The purpose of the present article is to review more recently published reports 
of research studies investigating the role of materials, and to evaluate whether 
EdReports.org has continued to accomplish the objectives of the change 
process. This study extends the findings reported in the previous article by 
examining two research questions. Reviews of published reports and policy 
papers, and perusal of a research study in progress are expected to 
demonstrate that a shift to the provision of high-quality materials, supported by 
professional learning, are critical effects for student learning. An evaluation of 
EdReports.org is expected to show that this organisation has disseminated and 
demonstrated its program successfully to key change agents, trained 
personnel, trialled the innovation, installed the program, and commenced 
assimilation of the program as an integral component of the American 
education system. 


Method 


The present study applies developmental research methodology to focus on 
progressive changes that have occurred in the findings of research studies into 
the effectiveness of materials and the status of the change process achieved by 
EdReports.org between 2016 and 2018. Content analysis of four recently 
published reports on research studies focused on shifts in meanings within the 
subject matter. A decision-oriented evaluation model was used for process 
evaluation to measure and interpret attainments during the program’s 
implementation to inform decisions pertaining to the procedural design. 


The procedures for collecting information for the study followed a sequence of 
three steps. First, links to reports on research studies and policy papers were 
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provided by two EdReports staff. Each report and policy paper was read to 
assess its relevance for content analysis. The reports, published by the 
Southern Regional Education Board, the Babson Survey Research Group, the 
RAND Corporation and the Center for American Progress, were selected for 
review, because they investigated the role of materials in relation to state or 
district adoption policies. The policy papers, published by the Aspen Institute, 
the Carnegie Corporation of New York, and Chiefs for Change, were selected 
for review, because they focused on the relationship between high-quality 
materials and the importance of professional learning. Second, published and 
unpublished articles provided by a key researcher were reviewed to document 
the scope and sequence of a current research study in progress. In addition, a 
70-minute video, Textbook Adoption in California: Issues and Evidence, 
presented by Policy Analysis for California Education, was viewed. Third, 
various documents, including a report on a case study, opinion papers, 
frequently asked questions and videos, were read or viewed on the 
EdReports.org, California Curriculum Collaborative and the Nebraska 
Instructional Materials Collaborative websites. Information about new state and 
district initiatives was read on the websites of the California Department of 
Education, Massachusetts Department of Elementary and Secondary 
Education, Mississippi Department of Education, Nebraska Department of 
Education, Rhode Island Department of Education, Tennessee Department of 
Education, Texas Education Agency, Wisconsin Department of Public 
Instruction, TNTP and Safal Partners. 


The procedure for analysing information, collected during the study, used 
content analysis method to summarise the subject matter focusing on 
interpretation of relevant documents. The reports of relevant research studies, 
policy papers and other documents were read and summaries prepared. 
Reporting the results involved organising the summaries chronologically, and 
incorporating them into the appropriate section of the article. During the course 
of the study, the first draft was sent to EdReports data strategist for review. 
Comments received from this person were incorporated into the section on 
EdReports.org. At the completion of the final draft, it was sent to an associate 
professor of education with expertise in curriculum alignment for review and 
comment. 


Research Studies on Instructional Materials 
Southern Regional Education Board 


In 2012, the Southern Regional Education Board began a multi-year study of 
how member states are implementing the CCSS. In March 2014 and January 
2015, the Southern Regional Education Board published two series of reports 
consisting of a summary report and five reports containing state profiles 
organised by five topics areas: timeline and approach to standards and 
assessments; aligned teaching resources; professional development; evaluation 
of teachers and leaders; and accountability. In May 2016, the Southern 
Regional Education Board published a summary report and 15 profiles on each 
state’s leadership efforts to support professional learning. 


In May 2017, the Southern Regional Education Board published a report on the 
alignment of materials, the subject of this review. In its report, the Southern 
Regional Education Board (2017) embraced highlights from states doing strong 
work, recommendations based on state trends, and state trends for each action 
area across the 2014-2015 and 2015-2016 school years. To guide data 
collection, Southern Regional Education Board researchers developed criteria 
for three action areas, each measured by three standards: minimal; essential; 
and strong. State education agencies in Alabama, Arkansas, Delaware, Florida, 
Georgia, Kentucky, Louisiana, Maryland, Mississippi, North Carolina, 
Oklahoma, South Carolina, Tennessee, Virginia and West Virginia participated 
in the study. Southern Regional Education Board researchers reviewed plans 
and reports on the state education agencies’ websites, interviewed state 
leaders about successes and challenges in the three action areas, interviewed 
educators in six states about their experiences with resources and services 
provided by their state education agencies, and collected feedback from each 
state education agency about draft state profiles. 


For the first action, establishing clear conventions for identifying high-quality, 
standards-aligned instructional materials, four states demonstrated strong 
efforts. Two recommendations, based on trends in state efforts, would address 
educators’ needs to verify publishers’ claims about alignment, differing levels of 
familiarity with standards and varying experiences in curriculum development, 
needs for tools to develop their own materials and needs for current materials. 
First, verify that criteria for developing and selecting materials fully and 
accurately reflect the content and rigour of the standards, and apply consistent 
criteria to evaluate textbooks and other materials. Second, use regular and 
frequent processes for involving educators in developing and _ selecting 
materials, and address educator needs for tools. Criteria for reviewing materials 
for alignment to standards, which the state education agency had verified as 
reflecting the content and rigour of the standards, had been established in nine 
states: Arkansas; Delaware; Florida; Georgia; Louisiana; Maryland; North 
Carolina; Oklahoma; and Tennessee. Verification, based on the use of 
nationally recognised rubrics or internally-developed criteria checked by 
external experts, was provided to state textbook committees in Florida, Georgia, 
Kentucky, North Carolina and Tennessee, and to districts in Arkansas, 
Delaware, Louisiana and Maryland. Criteria to assess alignment of online 
materials, posted on state education agencies’ repositories, to standards had 
been established in ten states: Alabama; Arkansas; Delaware; Florida; Georgia; 
Kentucky; Louisiana; Maryland; Mississippi; and North Carolina. Consistent 
criteria to review textbooks and items on state education agencies’ repositories 
had been established in nine states: Arkansas; Delaware; Georgia; Kentucky; 
Louisiana; Maryland; North Carolina; and West Virginia. In all 15 states, state 
laws established requirements for textbook selection and adoption. A state-level 
process was required in 11 states: Alabama; Florida; Georgia; Kentucky; 
Mississippi; North Carolina; Oklahoma; South Carolina; Tennessee; Virginia; 
and West Virginia. State laws did not provide for state-level adoption in four 
states: Arkansas; Delaware; Louisiana; and Maryland. In Louisiana, the state 
education agency conducts a curricular resources annotated review process. Of 
the 12 states that adopted or reviewed textbooks at the state level, the adoption 
cycle ensured that reviewers matched textbooks for alignment to standards in 
eight states: Louisiana; Mississippi for English Language Arts; North Carolina; 
Oklahoma; South Carolina; Tennessee; Virginia; and West Virginia. In the other 
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states, adoption cycles did not keep pace with changes in standards, usually 
because of delays due to lack of funds. Three degrees of authority for local 
selection of textbooks prevailed in all 15 states. Districts and schools were 
required to select textbooks from state-adopted lists in six states: Alabama; 
Florida; Kentucky; South Carolina; Tennessee; and West Virginia. Districts and 
schools were not required to select textbooks from state-adopted lists in five 
states: Georgia; Mississippi; North Carolina; Oklahoma; and Virginia. Districts 
and schools selected textbooks independently in four states: Arkansas; 
Delaware; Louisiana; and Maryland. In all 15 states, state education agencies’ 
staffs led or facilitated processes to develop, review and select materials for 
online repositories and collaborated with educators in these processes. The 
greatest variance among state education agencies related to the frequency of 
updating online repositories. 


For the second action, supporting local efforts to identify and use aligned 
instructional materials, six states demonstrated strong work. Two 
recommendations, based on trends in state efforts, would address educators’ 
needs for a set of resources that align specifically to state standards, and 
regular, sustained opportunities to study standards, instructional strategies, 
build skills in materials alignment, and collaborate with other educators. First, 
offer educators numerous and various guidance documents and aligned 
materials, and ensure that the state education agency's online repository is 
easy to find and navigate. Second, provide educators with integrated and 
sustained professional learning and technical assistance to support alignment 
efforts state-wide and tailor these services to fit local contexts. The most 
extensive guidance for building educators’ knowledge about aspects associated 
with materials was provided by six states: Alabama; Florida; Kentucky; 
Louisiana; Maryland; and North Carolina. Greater amounts of materials and 
high user-friendliness of state repositories were provided by the same six 
states. Integrated and intensive professional learning and technical assistance 
were provided by five states: Alabama; Delaware; Kentucky; Louisiana; and 
North Carolina. 


For the third action, using data to continuously improve state efforts, 11 states 
exhibited strong endeavours. One recommendation, based on trends in state 
efforts, would address the lack of rigorous research on the effectiveness of 
materials and professional development on materials alignment, and little 
evidence to guide policymakers’ decisions about improvement. Foster more use 
of research data, promote access to high-quality research on standards 
alignment, effectiveness of materials and professional development to align 
materials, and study the alignment and effectiveness of the state’s resources 
and services. In all 15 states, the state education agency collected at least one 
type from five types of data: patterns of educator use of state resources and 
services; educators’ perceptions of the quality of guidance, resources and 
support services; emerging needs of educators for aligned materials; impact of 
educator use of, and participation in, the state’s offerings; and local practices for 
developing and selecting materials. In all 15 states, data were gathered on a 
regular basis to improve leadership and support for materials alignment. Across 
all 15 states, data were most often used to inform decisions about improving the 
state education agency’s online repository and identifying ways to enhance 
professional learning and technical assistance services. 


Babson Survey Research Group 


The William and Flora Hewlett Foundation funded the Babson Survey Research 
Group to survey a sample of school districts to understand better the process by 
which materials were selected for Mathematics, English Language Arts, 
Science, and History and Social Studies. The study focused on identifying 
influences on the decision-making process and determining the degree to which 
district personnel were aware of and adopted open educational resources. The 
target population comprised randomly selected districts that provided schooling 
from kindergarten to grade 12. The districts, selected for the study, were sent 
an invitation and a questionnaire in the second quarter of 2017. The 
questionnaire consisted of two parts: the first part applied to all districts; and the 
second part intended to be completed only by districts that had made a 
selection decision for full-course materials over the previous three years. 
Responses were received from 584 districts representing 48 states and the 
District of Columbia. Approximately 79 percent of respondents were very 
familiar, 17 percent were somewhat familiar and four percent were unfamiliar 
with the adoption process within their districts. Only data collected from 
respondents, who were very familiar with the adoption process, were included in 
the data analysis. All data collected were entered into an online database for 
analysis in the form of percentile scores. 


Allen and Seaman (2017) reported on the nature of curricular decisions, the 
decision makers, factors driving selection, curricular sources, the decision- 
making process, awareness of licensing and open educational resources, and 
open educational resource adoptions. Full course curricular adoption decisions 
were common with 77 percent of districts making at least one decision over the 
past three years. While approximately 34 percent of these districts only adopted 
materials in one subject, 36 percent adopted materials in two subjects, 16 
percent adopted materials in three subjects, and 14 percent adopted materials 
in four subjects. Adoptions varied across subject areas with 59 percent of 
districts adopting materials for Mathematics, 44 percent adopting materials for 
English Language Arts, 29 percent adopting materials for Science, and 19 
percent adopting materials for History and Social Studies. The decision-making 
process involved a range of decision makers. Teachers participated in 93 
percent of districts, district administrators participated in 75 percent, principals 
participated in 73 percent, external experts participated in 21 percent, and 
parents participated in 10 percent. Respondents perceived that ten factors were 
critical in the decision-making process: 44 percent cited ‘comprehensive 
content’; 39 percent cited ‘works with our technology’; 39 percent cited ‘cost’; 20 
percent cited ‘supplemental materials’; 20 percent cited ‘adaptable materials’; 
14 percent cited ‘on state-approved lists’; 12 percent cited ‘includes test banks’; 
11 percent cited ‘easy to find’; seven percent cited ‘recommended by others’; 
and three percent cited ‘familiarity with brand’. Although materials marketed by 
12 main publishers were used in the districts, three publishers dominated the 
market: 46 percent cited Houghton Mifflin Harcourt; 42 percent cited Pearson; 
and 40 percent cited McGraw-Hill Education. An external factor was the primary 
driver for replacing materials with 69 percent of districts reporting that ‘changing 
standards’ was the main reason. Internal factors were less important with eight 
percent of districts reporting ‘regular review’, five percent reporting ‘teacher 
dissatisfaction’, and two percent reporting ‘district dissatisfaction’. Most districts 
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replaced the previous material after five years with 57 percent replacing the 
material after six to ten years, and 23 percent replacing the material after more 
than ten years. Most districts evaluated multiple alternatives with 27 percent 
considering three alternatives, 18 percent considering four alternatives, 15 
percent considering five alternatives, and 18 percent considering six or more 
alternatives. Most districts used a screening process to reduce a large, initial list 
of possible alternatives prior to a final evaluation of three alternatives in 42 
percent of districts or two alternatives in 26 percent of districts. The length of the 
decision-making process varied from one month to more than a year with 33 
percent of districts reporting that it took from six to nine months. Awareness of 
copyright and public domain was much higher among respondents than 
awareness of Creative Commons licensing. Approximately 65 percent of 
respondents reported some awareness of open educational resources, but this 
figure dropped to 34 percent, when awareness of licensing was included. 
Respondents had greater awareness of open educational resources than 
concepts and definitions. From a list of 14 open educational resources across 
four subject areas, respondents were asked to report whether they were aware 
of each resource, had included it for consideration, and whether it had been 
adopted. The overall level of awareness, consideration and adoption across 
these categories ranged from 50 percent to ten percent of districts. 


RAND Corporation 


In October 2016, the RAND Corporation surveyed a randomly selected, 
nationally representative sample of 1,349 school leaders, who periodically 
received surveys on education issues of national importance, on_ their 
perceptions of curriculum requirements and knowledge of standards. 
Specifically, the objectives of the survey were to identify what materials were 
required or recommended for English Language Arts and Mathematics, how 
school leaders’ knowledge of standards compared with teachers’ knowledge 
identified in a previous survey, and what factors may explain school leaders’ 
knowledge of standards. Of the sample, 422 school leaders responded to the 
survey. Data collected from the respondents were weighted to ensure that the 
proportionate representation of school leader backgrounds and demographic 
characteristics matched available national data. Raw data were analysed to 
produce percentile scores for the first two objectives, but regression was used 
to summarise the relationship between two variables for the third objective. 


Kaufman and Tsai (2018) reported the results of the survey across three 
objectives. Of the ten most commonly required or recommended materials for 
English Language Arts, 64 percent of elementary leaders and 22 percent of 
secondary leaders cited levelled readers. A high proportion of elementary 
leaders reported using two guided reading series: 42 percent cited RAZ-Kids 
published by Learning A-Z; and 35 percent cited Accelerated Reader published 
by Renaissance Learning. Trade books were also cited by 49 percent of 
elementary leaders. Most of the English language arts materials had not been 
reviewed by EdReports.org. Of the ten most commonly required or 
recommended materials for Mathematics, ten percent or more elementary 
leaders reported using four textbooks, while ten percent or more secondary 
leaders reported using two textbooks. All ten of the mathematics materials had 
been reviewed by EdReports.org, but only one material met expectations for all 
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categories at all grade levels. Twenty-six percent of elementary leaders and ten 
percent of secondary leaders used English language arts materials that met 
EdReports expectations, but none of these materials were among the ten most 
commonly required or recommended materials. Twenty percent of elementary 
leaders and ten percent of secondary leaders used mathematics materials that 
met EdReports expectations with most using the one cited among the ten most 
commonly required or recommended materials with few using two other 
materials that met EdReports expectations. A high proportion of leaders were 
unable to identify reading approaches most aligned to state standards for 
English Language Arts, a situation also reflected among teachers. Leaders 
were able to identify key aspects of rigour in mathematics standards at about 
the same rate as mathematics teachers. Regression analysis showed that 
leaders, who reported requirements or recommendations to use standards- 
aligned materials for English Language Arts, were better able to identify aligned 
reading approaches. 


The results suggested that only a small proportion of schools had requirements 
or recommendations for materials with strong evidence of alignment to the 
CCSS. The main implication of this finding for policy and practice is that state 
and district leaders should consider the messages they are sending to teachers 
about requirements or recommendations for materials. For instance, state and 
district leaders should examine how their review processes reflect those used 
by EdReports.org, since this organisation’s review criteria align with the CCSS. 
State and district leaders should support principals through professional 
development, especially at the elementary level given the high use of levelled 
and guided readers, and at the secondary level, where mathematics content is 
becoming more sophisticated. 


Center for American Progress 


Center for American Progress researchers surveyed the use of materials for 
English Language Arts and Mathematics in grades 4 and 8 by the 30 largest 
school districts to identify what materials districts were using and whether these 
materials were highly rated. Data about the materials were identified on the 
districts’ websites and then followed up by email for confirmation, or further 
information in cases where the data were unavailable on websites. The ratings 
for each material, identified from searches of the rating systems maintained by 
EdReports.org and the Louisiana Department of Education, were reported by 
district for English Language Arts and Mathematics. The researchers contacted 
district personnel and technical assistance providers to develop case studies on 
the adoption and implementation processes in two districts that had adopted 
highly rated materials. A case study reported on another district, not included in 
the sample, was based on a published report. 


Partelow and Shapiro (2018) reported the results of the analysis, outlined case 
studies on three districts, and presented five recommendations for district 
leaders. Only 19 districts posted information about adopted or recommended 
materials online and only 17 districts showed the adoption process online. Two 
districts used district-created materials and several other districts used 
materials that had not been reviewed by EdReports.org or Louisiana’s review 
systems. Two districts failed to provide information about adopted or 
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recommended materials. Of 26 districts, whose materials were known, 11 
districts did not use any highly rated materials and 24 districts were using or 
recommending at least one material with a low rating. Only 29 percent to 35 
percent, depending on the subject or grade, of the materials that districts 
reported adopting or recommending were highly rated by EdReports.org. Only 9 
percent to 20 percent, depending on the subject or grade, of the materials that 
districts reported adopting or recommending were highly rated by the Louisiana 
Department of Education. Shelby County Schools in Tennessee, Duval County 
Public Schools in Florida, Wake County Public School System in North Carolina 
and Jefferson County Public Schools in Kentucky stood out as districts having 
adopted or recommended materials that were highly rated across most subjects 
and grades. The case study on the Wake County Public School System focused 
on district leaders using EdReports reviews to rate open educational resources, 
screen the reviews to provide a list of recommendations for community input 
sessions, adopt two free open educational resources for English Language Arts 
and Mathematics, and provide professional development for teachers to support 
implementation. The case study on Duval County Public Schools focused on 
district leaders’ decisions to purchase Eureka Math published by Great Minds 
and provide professional development to ensure teachers understood its 
content. Evidence that this approach led to improvement in student 
achievement encouraged Duval County Public Schools and four neighbouring 
districts to create professional learning communities built around four-week 
cycles of content-embedded professional learning. 


Partelow and Shapiro presented five recommendations for district leaders. First, 
they should create systems and develop policies that facilitate and incentivise 
good choices. Second, they should examine and solve barriers to adoption. 
Third, they should provide teachers with professional development for 
implementing adopted materials. Fourth, they should increase transparency by 
publishing information about the adoption process and adopted materials on 
district websites. Fifth, they should verify the effectiveness of materials with 
learners. 


Policy Papers 
Aspen Institute 


In April 2017, the Aspen Institute’s Education and Society Program released a 
policy paper arguing for integration of materials and professional learning. In 
June 2017, the Aspen Institute held a panel discussion focusing on how to 
connect materials and professional learning in schools. Wiener and Pimental 
(2017) contended that curriculum and _ professional learning activities 
undertaken by the Louisiana Department of Education, the District of Columbia 
Public Schools and a regional education service agency in West Virginia 
illustrated positive change efforts. Key considerations and enabling conditions 
for system leaders to organise professional learning and high-quality materials 
were outlined in six recommendations. First, the quality of materials is 
important. Second, content-specific inquiry cycles improve practice. Third, the 
effects of strong collaborative culture in schools are powerful. Fourth, teachers 
need time to improve instruction. Fifth, content experts should facilitate 
professional learning. Sixth, system leaders have vital roles and responsibilities. 
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Carnegie Corporation of New York 


In July 2017, the Carnegie Corporation of New York released a policy paper 
examining the availability of aligned materials in relation to the NGSS, outlining 
the importance of integrating professional learning into science education, and 
presenting a vision for providing high-quality materials. Bybee and Chopyak 
(2017) found that the current supply of materials is disaggregated and difficult to 
find, none have been evaluated, and development of new materials by 
publishers is uncoordinated and siloed. Developers of new materials need to 
take into account five innovations inherent in the NGSS: explaining phenomena 
and designing solutions; integrating three dimensional learning; building 
programs across grade levels; aligning with English Language Arts and 
Mathematics; and linking materials to sustainable and continuous professional 
learning. A vision for implementing the NGSS includes incorporating materials 
as an essential component, professional learning linked with materials, 
collaboration between actors in the marketplace, integration by field testing 
materials, differentiating high quality, better coordination between actors to 
address innovations, integrated professional learning, equity and access for all 
students and teachers, and high-quality instruction and assessment. Realisation 
of this vision can be met by four solutions: expand the supply of high-quality, 
aligned materials; increase public understanding about the benefits of materials; 
innovation to accelerate implementation; and reduce system challenges by 
leveraging opportunities. 


Chiefs for Change 


In August 2017, Chiefs for Change, a bipartisan alliance of state and district 
superintendents, released a policy paper advocating for the use of six strategies 
to ensure that a standards-based curriculum is matched with high-quality 
materials. Citing efforts at curriculum reform in Massachusetts, New York, 
Louisiana and districts in Florida, Chiefs for Change (2017) issued six 
recommendations for state and district leaders. First, use incentives, not 
mandates, to maintain local authority. Second, emphasise evidence, and start 
small, if the research base has not been developed. Third, leverage teachers’ 
expertise and teacher leaders in this work. Fourth, use the procurement process 
to expand use of the highest-quality materials. Fifth, create professional 
learning focused on curricular content. Sixth, messaging matters, and external 
partners and validators can help. 


Research Study in Progress 


Researchers, Morgan Polikoff from the University of Southern California and 
Cory Koedel from the University of Missouri, initiated a study into the patterns of 
textbook adoptions in California, Illinois, Texas, New York and Florida, the effect 
of the CCSS implementation on changing adoption patterns, the ways districts 
make adoption decisions, teachers’ use of textbooks and the impact of 
textbooks on student outcomes. In December 2014, funds were received to 
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commence a project to investigate textbook adoptions and their effects on 
student achievement in California. 


In 2000, 100 San Francisco County students filed a lawsuit against the State of 
California for failing to provide public school students with equal access to 
materials, safe and decent school facilities, and qualified teachers. Settled in 
2004, the Eliezer Williams and others versus the State of California case led to 
the state allocating additional funds to rectify deficiencies in schools. As part of 
the Williams settlement, every local board is required to hold an annual public 
hearing to determine whether each student has sufficient basic materials in the 
core subject areas that are aligned to the state standards. Legislation enacted 
in 2007 requires a record of the textbook in each core subject, reported by each 
school, to be maintained by the California Department of Education on school 
accountability report cards. Over two years, a team of students at the University 
of Southern California compiled data on mathematics textbooks from the school 
accountability report cards for 7,600 elementary and middle schools. 
Approximately 240 mathematics textbooks were identified, but only four 
textbooks, adopted in 1,878 schools, provided sufficient power for statistical 
analysis. Information on textbook adoptions was merged with data on 
demographic characteristics for these schools and achievement outcomes 
using test scores on state mathematics assessments for grade 3. Koedel, Li, 
Polikoff, Hardaway and Wrabel (2017) found that California Math, published by 
Houghton Mifflin, was more effective than the other three textbooks in 
increasing student achievement over three years during grades 3, 4 and 5. 


From 2015 to 2017, the researchers interviewed district leaders to gain an 
understanding of the evaluation and adoption process to inform how textbook 
quality could affect the adoption process. A random sample of 34 district 
leaders, selected for the interviews, was stratified according to the type of 
mathematics textbook, school size and level of student achievement. A semi- 
structured interview schedule, focusing on the adoption process, quality and 
alignment of materials, and teacher use of materials, was used in the 
interviews. NVivo, a qualitative data analysis package, was used to code the 
frequency of certain elements of the adoption process using a data matrix. 
Campbell and Polikoff (2017) found that district leaders described similar 
adoption processes characterised by two phases: an initial screening phase; 
and an evaluation phase. During both phases, the adoption process was 
characterised by institutionalised elements with differences between districts 
arising from specific factors, such as, the proportion of English language 
learners, district size, and access to technology. The screening process was 
similar, irrespective of whether materials were selected from the state-adopted 
list or off the state list. There were differences, however, in the attitudes of 
leaders from districts that considered materials on the state-adopted list and 
those, who considered off-list materials. In the former case, leaders believed 
that the state adoption process identified materials aligned to the state 
standards, while in the latter case, leaders cited concerns about the quality and 
rigour of the state adoption process. Timing of the decision to start the adoption 
process was another important factor. Those districts that started early had 
limited choice from textbooks that publishers had relabelled, CCSS-aligned, 
while districts that started adoption later were generally satisfied with a wider 
choice of CCSS-aligned materials. The process for evaluation involved forming 
a representative committee, applying criteria reflecting the needs of the district, 
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evaluating two or three materials, and sometimes piloting materials in 
classrooms. Differences in the complexity of the evaluation process could be 
attributed to the size of the district. The county office of education played an 
important intermediary role between the state and local levels, but this role 
differed depending on the size of the district. For larger districts, the county 
office organised publisher displays and trained district committees. For smaller 
districts, the county office facilitated collaboration between neighbouring 
districts. The evaluation process, however, did not reliably select a high-quality 
material, since the guiding principle was to choose a material that satisfied 
consensus between teachers. 


EdReports.org 
Origins 


EdReports.org was founded in 2014 following a series of meetings of the 
Mathematics Strategy Group held in 2012 and 2013 at the Annenberg 
Foundation Trust at Sunnylands in Rancho Mirage, California. Concluding that 
an entity should be created to review materials, grants were received from 
several philanthropic institutions to establish a non-profit organisation. A 
Seattle-based consulting group, Education First, contracted by the funders to 
plan the new organisation, recruited a Board of Directors, developed a business 
plan, hired an executive director, housed the organisation, provided technical 
support and launched a website at www.edreports.org. 


Review and selection process 


EdReports staff analysed 11 commonly used rubrics, and observed review 
processes and training conducted for Achieve and the state of Tennessee to 
develop a process for reviewing digital and print-based materials. The review 
process requires materials to meet criteria and indicators set for three 
successive gateways: alignment to the CCSS; rigour of the subject matter; and 
instructional supports and usability. 


To identify materials for review, EdReports staff researches the marketplace, 
accepts recommendations from educators and receives requests from 
publishers. The review process involves teams of four or five reviewers 
evaluating each material independently and providing evidence through the 
online system before meeting to reach consensus on the evidence and the 
score. Review teams are assisted by volunteer advisory panels with expertise in 
each content area. 


Mathematics 


Early in 2014, EdReports staff conducted a listening tour to collect information 
for developing a methodology and a tool to review mathematics materials for 
kindergarten to grade 8. An Anchor Educator Working Group analysed the 
feedback and developed the Quality Instructional Materials Tool: K-8 
Mathematics and K-8 Mathematics Evidence Guides. Following a calibration 
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exercise to ensure consistency across reviewers, 20 materials for kindergarten 
to grade 8 were evaluated by 47 reviewers. EdReports.org posted the results in 
March 2015. 


Following criticism of the gateway procedure by several publishing companies 
and release of an open letter by the National Council of Teachers of 
Mathematics and the National Council of Supervisors of Mathematics outlining 
concerns about the review process, EdReports.org released four enhancements 
to the review tool, methodology and reporting protocol. Then, the review teams 
reviewed the materials again as well as an additional 58 materials. Evaluations 
of these materials were released on a rolling basis in February, April and May of 
2016. 


Early in 2015, EdReports.org conducted a listening tour to collect information for 
developing a methodology and tool to review mathematics materials for high 
school. An Anchor Educator Working Group analysed the feedback and 
developed the Quality Instructional Materials Tool: High School Mathematics 
and High School Mathematics Evidence Guides. Late in 2015, 31 reviewers 
evaluated eight traditional course materials for algebra |, geometry and algebra 
Il as well as integrated course materials. EdReports.org released the results for 
four traditional course materials and one integrated course material in June 
2016. After reviewing an additional nine materials, EdReports.org posted the 
results in October 2016. 


English Language Arts 


In mid-2015, EdReports.org conducted a listening tour to collect information for 
developing a methodology and tool to review English language arts materials 
for grades 3 to 8. An Anchor Educator Working Group analysed the feedback 
and developed the EdReports.org Quality Instructional Materials Tool: ELA 3-8 
Review Tool and 3-8 ELA Quality Instructional Materials Tool Evidence Guides. 
Late in 2015, 45 educators reviewed seven English language arts materials for 
grades 3 to 8. In August 2016, EdReports.org posted the results. In September 
2016, EdReports.org held a webinar to report the findings of the review. 


In 2016, EdReports.org conducted a listening tour to collect information for 
developing a methodology and tool to review English language arts materials 
for kindergarten to grade 2. An Anchor Educator Working Group analysed the 
feedback and developed the EdReports.org Quality Instructional Materials Tool: 
ELA K-2 Review Tool and K-2 ELA Quality Instructional Materials Tool 
Evidence Guides. Early in 2017, review teams reviewed six English language 
arts materials for kindergarten to grade 2. In April 2017, EdReports.org posted 
the results. 


In 2017, EdReports.org conducted a listening tour to collect information for 
developing a methodology and tool to review English language arts materials 
for high school. An Anchor Educator Working Group analysed the feedback and 
developed the EdReports.org Quality Instructional Materials Tool: ELA High 
School Review Tool and High School ELA Quality Instructional Materials Tool 
Evidence Guides. Late in 2015, six review teams reviewed six English language 
arts materials for high school. EdReports.org posted the results in August 2017. 
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California initiatives 


In mid-2015, EdReports.org began working with Orange County Department of 
Education, which provides support services to 27 districts in southern California. 
Mathematics coordinators, Jody Guarino and Vanessa Cerrahoglu offered to 
become partners in an effort to provide better guidance and support using 
EdReports ratings by developing a series of trainings and supports delivered to 
three districts. In September 2015, superintendents, coaches and teachers from 
three districts and one charter school met in Westminster to use EdReports 
ratings to screen available materials and pilot selected materials in classrooms. 


One of these districts, Newport-Mesa Unified School District, consisting of 32 
schools located in Newport Beach, Costa Mesa and Corona del Mar, asked 
Orange County Department of Education for more intensive support to adopt a 
new mathematics material. Guarino, Cerrahoglu, Drake and Weisskirk (2018) 
reported that the adoption process including professional learning consisted of 
three phases: ground the work in shared understanding of the standards; apply 
the learning to two cycles of materials evaluation; and build consensus. A 
steering committee was trained to screen available materials, publishers were 
involved in the final selection of two materials, 30 lead pilot teachers were 
trained, and 123 teachers piloted the materials. Since evidence collected during 
the pilot was inconclusive, a consensus process was used to reach agreement 
about adoption of one material. The teacher-centred approach embodied in the 
decision-making process assisted implementation of the adopted material. 
Afterwards, EdReports staff wrote a document, Building Capacity and 
Consensus through a Teacher-led Materials Adoption: a Case Study from 
Newport-Mesa Unified School District. 


Based on the success of the collaborative work with Orange County 
Department of Education, EdReports.org decided to customise its work to meet 
the needs of California's large materials’ marketplace and the provision, 
introduced in 2013, allowing districts to use materials not adopted by the State 
Board of Education. At a grantee conference held by the William and Flora 
Hewlett Foundation, EdReports Executive Director, Eric Hirsch, met staff of 
Pivot Learning, a non-profit organisation based in the San Francisco Bay Area. 
EdReports.org formed a partnership with Pivot Learning to provide independent 
reviews to help teachers identify high-quality materials for use in California 
schools. The two organisations raised funds for the venture in 2016, and 
developed the components for a new website, California Curriculum 
Collaborative, launched at www.calcurriculum.org in February 2017. Following 
the launch, the two organisations hosted a series of regional workshops to 
guide district teams to use the tool and processes on the website. Funded by 
the Bill and Melinda Gates Foundation, the two organisations added guidance 
to the website for Science and English language learners. In 2018, workshops 
were conducted for districts with high proportions of students from low socio- 
economic and minority backgrounds. Early in 2019, workshops were held to 
discuss the 2018 state adoption of science materials and EdReports reviews of 
science materials. 
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The California Curriculum Collaborative is organised into six areas. About 
presents information about the partnership between EdReports.org and Pivot 
Learning, and the findings of research studies about the effects of materials on 
student learning. News presents news articles about the California Curriculum 
Collaborative. Resources present links to professional resources. Reports set 
out EdReports review process, annotations for kindergarten to grade 8 and high 
school materials for English Language Arts and Mathematics containing 
EdReports reviews and indicators of state adoption, and a timeline for 
publishing EdReports reviews of science materials and the 2018 state adoption 
of science materials. Implementation presents Refining Implementation: a 
Guide for Instructional Materials in the Field, a document setting out a six-step 
implementation process consisting of pre-work, discover, prioritise, ideate, plan 
and prototype, and support for implementing mathematics materials. Adoption 
presents key questions and guidance for adopting materials for English 
Language Arts and Mathematics at the local level, and the 2018 state adoption 
of science materials. 


Professional learning 


Learning Forward, a professional association based at Oxford, Ohio, which 
builds the capacity of leaders to establish and sustain effective professional 
learning, focused on strengthening professional learning for implementing 
materials at its summer institutes held at Portland, Oregon, in July 2018. 
Learning Forward collaborated with EdReports.org, the Carnegie Corporation of 
New York, BSCS Science Learning, CenterPoint Education Solutions, the 
Institute for the Study of Knowledge Management in Education and UnboundEd 
to facilitate the summer institutes. 


Prior to the institutes, Learning Forward (2018) published a paper exploring the 
premise that good teachers can motivate and engage students by using high- 
quality materials. Three examples of exemplary practice were cited to support 
this contention. A case study on the implementation of EL Education’s K-5 
Language Arts Curriculum at Hollis Innovation Academy in Atlanta used 
professional learning communities, learning walks to help teachers make 
practice shifts, strategy cafes to provide opportunities for teachers to participate 
in differentiated learning, and a data room for using data for improvement. A 
case study on the implementation by Colonial School District in Delaware of 
Connected Mathematics Program, published by Pearson, used district-wide 
professional learning and job-embedded learning at each school site, 
instructional rounds to measure teachers’ cognitive growth targets, and 
collected evidence about the impact of professional learning. A case study on 
the piloting of Core Knowledge Language Arts published by Core Knowledge 
Foundation, Wit & Wisdom published by Great Minds and the Read Aloud 
Project published by Student Achievement Partners by 13 districts in 
Tennessee used supports provided by TNTP, participation in an early literacy 
program, Read to be Ready, and the Instructional Practice Guide to observe 
lessons. Five lessons were learnt from evidence obtained in the case studies. 
First, selecting high-quality, aligned materials are of key importance. Second, 
using a standards-aligned material well requires skilful professional learning. 
Third, investing in leadership at the school and district levels is essential. 
Fourth, ensuring expert teacher leaders is important. Fifth, effective team 
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learning is part of a larger instructional improvement and learning system. 
Connecting professional learning to high-quality materials, however, poses 
three challenges: aligning assessments, observations and_ curriculum; 
establishing sufficient regularly scheduled time and structure for professional 
learning communities and other teaching strategies; and applying change 
management strategies. Five actions were recommended to_ integrate 
professional learning and high-quality materials. First, build deeper knowledge 
about this issue. Second, assess the quality of the curriculum. Third, establish 
professional learning communities. Fourth, strengthen learning teams. Fifth, 
develop building- and team-level expertise. 


In May 2018, Learning Forward held a webinar, Building the Case for 
Connecting High-Quality Curricula and Team-Based Professional Development. 
Presenters, Stephanie Hirsh and Tracy Crow of Learning Forward, Eric Hirsch 
and Lauren Weisskirk of EdReports.org, and Jody Guarino of Orange County 
Department of Education discussed research findings confirming the 
importance of aligned curriculum and materials, and using professional learning 
communities to ensure effective implementation. 


The four-day institutes were conducted by 21 facilitators from the six 
organisations as well as Ed Trust and Cherry Creek School District in Colorado. 
The program consisted of an introductory session attended by 227 participants 
on the first day, the choice of one of five sessions on the second and third days 
conducted by a partner, and a concluding session for all participants on the last 
day. The introductory session focused on advancing equity and excellence 
through professional learning tied to curriculum implementation. EdReports.org 
focused on high-quality materials and aligned professional development. BSCS 
Science Learning focused on building capacity to implement the NGSS. 
UnboundEd focused on understanding the standards and shifts in secondary 
mathematics. The Institute for the Study of Knowledge Management in 
Education focused on adopting open educational practice to improve literacy- 
based curriculum at the secondary level. CenterPoint Education Solutions 
focused on strengthening assessment literacy. In the concluding session, 
Learning Forward focused on how team learning cycles and _ high-quality 
materials strengthen learning. 


Science 


Late in 2017, EdReports.org conducted a listening tour to collect information for 
developing a methodology and tool to review science materials for grades 6 to 
8. In January 2018, an Anchor Educator Working Group analysed the feedback 
and developed the rubric and evidence guides. Beginning in May 2018, review 
teams reviewed six science materials for grades 6 to 8. EdReports.org expects 
to post the results early in 2019. 


New projects 
The initiatives in California led EdReports.org to form partnerships in two other 


states. In 2014, TNTP partnered with three districts in Florida to assess their 
implementation of the Florida Standards. Collection and analysis of data 
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identified challenges to implementation arising from inconsistent professional 
development, and misaligned curricula and materials. In 2016, TNTP formed 
the Pilot Florida Implementation Network with six districts: Brevard; Broward; 
Duval; Highlands; Pasco; and Pinellas. The six districts are collaborating by 
establishing network goals, sharing work, and designing individual district work 
focusing on a few key areas. Focus on materials led the districts to request 
EdReports.org to support district reviews of mathematics materials under 
consideration for state adoption in 2018. EdReports.org released reviews of 
Florida editions for 15 mathematics materials between January and June of 
2019. In October 2018, the Texas Education Agency contracted Safal Partners, 
TNTP and EdReports.org to conduct the Instructional Materials Quality 
Evaluation Pilot intended to design an online Instructional Materials Portal. Safal 
Partners, an education management firm located at Houston, and TNTP are 
overseeing independent reviews of English language arts and reading materials 
conducted by Texas educators. EdReports.org is designing a customised rubric 
to be piloted with Texas educators in May 2019. Beginning in November 2019, 
the reviews will be posted on the Instructional Materials Portal. 


In 2016, CCSSO’s Chief Academic Officer Network met in Louisiana and 
examined the Louisiana Department of Education’s initiative to assist districts 
ensure that teachers use high-quality materials. Network members visited 
schools, interviewed administrators and teachers, and learnt about Louisiana’s 
policy to provide districts and schools with detailed information and tiered 
rankings of materials and professional development providers. The Chief 
Academic Officer Network decided to adapt this model for use in their own state 
settings and asked CCSSO to support them. In February 2017, CCSSO created 
the Instructional Materials Professional Development Network to support 
Delaware, Massachusetts, Mississippi, Nebraska, New Mexico, Rhode Island, 
Tennessee and Wisconsin initiate state projects. CCSSO periodically convenes 
meetings for teams from the eight states to provide opportunities for cross-state 
collaboration, discussions with experts, feedback on state plans and challenges, 
and time to work on state plans. 


The Massachusetts Department of Elementary and Secondary Education is 
collaborating with Teach Plus and the Rennie Center for Education Research 
and Policy to convene panels of Massachusetts educators early in 2019 to 
review existing research and evidence about core curriculum materials. 
Contracted to support the Curriculum Ratings by Teachers project, 
EdReports.org presented webinars on EdReports review process and is 
consulting on a publishers’ strategy. Working in teams at panel sessions, the 
educators will evaluate, calibrate and rate the materials, and conduct document 
reviews and synthesis independently between panel sessions. By the end of 
three months, the teams will reach agreement on the materials’ alignment with 
the Massachusetts Curriculum Frameworks, and then share their findings with 
state curriculum leaders during two meetings. The outcome of the project will 
consist of a series of reports to inform selection decisions by Massachusetts 
educators. In 2018, the Mississippi Department of Education began the High- 
Quality Instructional Materials and Professional Learning Initiative for 18 
districts to participate in a three-year pilot consisting of intensive training in the 
Mississippi College- and Career-Ready Standards, reviewing the alignment of 
mathematics materials with the standards, and developing a list of mathematics 
materials and professional learning. EdReports.org was contracted to design a 
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rubric, train mathematics reviewers and support the pilot districts. The Nebraska 
Department of Education contracted EdReports.org to design a state-specific 
website featuring EdReports reviews and collaborate with Nebraska educators 
to develop guidance documents for adopting and using materials. Launched in 
August 2018 at www.nematerialsmatter.org, the Nebraska Instructional 
Materials Collaborative is organised into five areas. ELA presents EdReports 
reviews searchable by grade, eight steps for selecting materials, and links to 
external resources. Math presents EdReports reviews searchable by grade, 
eight steps for selecting materials, and links to external resources. Science 
presents links to external resources. Nebraska Resources defines key terms, 
sets out eight steps for selecting materials and links to selection resources, and 
presents a_ toolkit consisting of communications resources, reports and 
frequently asked questions. About presents a rationale statement for the 
Nebraska Instructional Materials Collaborative and contact details. Following 
the launch, EdReports.org is training the staffs of the state’s 19 educational 
service units to support districts to use the website. The New Mexico Public 
Education Department contracted EdReports.org to review the state adoption 
process. In June 2018, EdReports.org audited the work of the summer review 
institute, wnen New Mexico educators evaluated the alignment of science and 
arts materials with the New Mexico Content Standards. EdReports.org provided 
recommendations for improving the review criteria and adoption process for the 
Mathematics adoption in 2019 and English Language Arts adoption in 2020. 
After working with Rhode Island districts in 2016, the Rhode Island Department 
of Education invited EdReports.org to participate in its Honours Colloquium. In 
January 2018, Eric Hirsch and Lauren Weisskirk presented an overview on 
EdReports work to Division of Teaching and Learning staff. Following the 
presentation, the Rhode Island Department of Education contracted 
EdReports.org to assist 16 districts access and use information about the 
quality of mathematics and science materials during the adoption process. The 
Tennessee Department of Education contracted EdReports.org to develop a 
train-the-trainer model for staff of the state’s eight centres of regional excellence 
to support district materials selection for the state adoption process in 2019. 
The first training sessions were delivered in October and December of 2018 
with subsequent sessions scheduled in 2019. In November 2018, the Wisconsin 
Department of Public Instruction hosted a one-day professional learning event, 
Quality Instructional Materials: Unlocking Teacher Creativity and Increasing 
Equity, to promote the use of high-quality materials supported by curriculum- 
based professional learning. Eric Hirsch and Kate Gerson of UnboundEd 
presented the workshop to 1,200 educators consisting of school-based teams at 
the state’s 12 cooperative educational service agencies. 


Organisational productivity 


By January 2019, EdReports.org had reviewed nearly 500 textbooks, digital 
materials and open educational resources for English Language Arts and 
Mathematics. Five hundred and twenty districts across the USA, enrolling over 
7,600,000 students, use EdReports reviews to select materials. 


In October 2018, EdReports.org launched a revamped website, which is 


organised into five areas. Compare Materials presents the reports of reviews in 
English Language Arts for kindergarten to grade 2, grades 3 to 8, and high 
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school and Mathematics for kindergarten to grade 8, and high school in a form 
that allows users to compare text quality, building knowledge, alignment rating 
and usability rating. Reports Center presents the reports of reviews as a 
database searchable by publisher, subject grade or report title. Resources 
present various resources relating to EdReports activities. Impact presents 
information about productivity and outcomes for reviewers. About Us presents 
biographies about EdReports staff, Board of Directors and reviewers, the review 
process, recruitment of staff and reviewers, frequently asked questions, and 
contact details. 


Organisational structure 


EdReports.org has established a middle-sized organisation with grants provided 
by the Bill and Melinda Gates Foundation, Broadcom Incorporated, Carnegie 
Corporation of New York, Charles and Helen Schwab Foundation, Charles and 
Lynn Schusterman Family Foundation, Leona M. and Harry B. Helmsley 
Charitable Trust, Oak Foundation, Overdeck Family Foundation, Samueli 
Foundation, Stuart Foundation, and William and Flora Hewlett Foundation. 
EdReports.org is governed by a twelve-member Board of Directors, whose 
members have professional backgrounds in education, finance, law, non-profit 
management, and marketing and communications. 


EdReports.org is operated by 27 staff members consisting of an executive 
director, a chief operating officer, a chief strategy officer, two outreach 
specialists, a human resource manager, an office manager, two program 
managers, a program coordinator, an operations coordinator, three 
communications specialists, two instructional materials managers, three 
directors of review, six content specialists, an early literacy consultant, and a 
data strategist. EdReports.org expects to hire up to ten new staff members in 
2019. EdReports.org is a virtual organisation with staff working in home offices 
using communications technology to support collaboration. Staff members, 
however, meet at annual retreats as well as board meetings, workshops and 
reviewer trainings. 


EdReports.org maintains a network of 375 content reviewers representing 
education systems in urban, suburban and rural communities across 46 states. 
After completing an interview and a work sample, prospective reviewers are 
selected according to depth of content knowledge, experience at evaluation, 
and ability to participate in face-to-face meetings and virtual conferences, but 
they must not be affiliated with the publishing industry. Each summer, newly 
recruited reviewers participate in annual training sessions, which were held in 
Chicago from 2014 to 2017 and in Minneapolis in 2018. Each reviewer reviews 
materials independently, but meets regularly in small groups to discuss the 
findings. Reviewers are paid a stipend at the end of each review. 


Discussion 
The results of this study show that the current wave of research studies 


investigating the role of materials and consequential initiatives to improve 
evaluation and selection of materials, arising from implementation of the CCSS 
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and NGSS, have prospered with the publication of new reports on research 
studies, the release of several policy papers and the commencement of new 
projects by EdReports.org. 


The research studies covered a range of issues relating to selection and 
adoption of materials at the state and local levels. The Southern Regional 
Education Board (2017) investigated conventions for identifying high-quality, 
standards-aligned materials, local efforts in identifying and using aligned 
materials, and use of data to improve state efforts. The Louisiana Department of 
Education’s Curricular Resources Annotated Review process was rated highly, 
because it assesses alignment of textbooks and benchmark assessments to the 
Louisiana State Standards. Involving public comment, screening of materials, 
review and ranking of materials by teachers on a rolling basis, the process 
provides local leaders and teachers with reliable information to select high- 
quality materials. Consequently, Louisiana teachers show higher levels of 
knowledge about their state standards than nationally, and use materials and 
teaching strategies aligned to standards at a higher rate than in other states. 
The Teacher Support Toolbox provides online curricular resources to support 
instruction, and district support networks assist district leaders build the capacity 
of teachers in the curriculum. These efforts contributed to notable advances that 
Louisiana teachers have made in knowledge and practice. Allen and Seaman 
(2017) investigated the decision-making process for selecting materials. They 
found that almost all adoption decisions are driven by a belief that current 
materials no longer meet current standards. Kaufman and Tsai (2018) 
investigated school leaders’ perceptions about adopted materials and their 
knowledge of standards. They found that many adopted or recommended 
materials, reported by school leaders, were not aligned to state standards, and 
school leaders failed to identify reading approaches and mathematics topics at 
higher grade levels. The Center for American Progress (2018) investigated 
reporting by large districts of adopted materials and their use of high-quality 
materials. They found that most large districts have not adopted highly rated 
materials and failed to provide information to the public about the adoption 
process or adopted materials. 


Each of the policy papers uses the findings of current research studies as a 
foundation to advance particular policy positions consisting of assumptions 
about the nature of school systems, knowledge and values. Wiener and 
Pimental (2017) used research findings that most professional learning does not 
meet teachers’ needs, many teachers do not have access to high-quality, 
standards-aligned materials, professional learning needs to address teachers’ 
decisions within the classroom context and teachers learn best through a 
collegial approach to advance a policy position that implementation of materials 
needs professional learning support. Bybee and Chopyak (2017) used research 
findings about the textbook publishing industry to advance the policy position 
that high-quality materials are essential to support implementation of the NGSS. 
Chiefs for Change (2017) used research findings showing lack of alignment 
between standards and materials, which publishers claim to be standards- 
aligned, and the importance of a content-based curriculum to advance the 
policy position that content-rich, standards-aligned and high-quality materials 
affect student achievement. 
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The large-scale study on patterns of textbook adoptions in five states is still at a 
prefatory stage, but promises to provide research-based evidence that materials 
represent a key factor affecting education reform. In an opinion paper 
discussing impressions concerning progress of the study, Polikoff (2018) 
reported on challenges collecting and analysing textbook adoption data, 
delivering high-quality materials to teachers, and getting teachers to use them. 
Resistance to the collection of textbook data, difficulty in identifying sources for 
such data, problems in determining textbook editions, and the appropriateness 
of data analysis strategies were cited in relation to the first challenge. The 
decentralised nature and _institutionalised practices of local selection 
procedures, and time delays in disseminating information about effective 
textbooks to district leaders were cited in relation to the second challenge. 
Teachers’ widespread use of a variety of materials was cited in relation to the 
third challenge. 


Planning decisions to create an entity to review materials were activated by 
conversations within the Mathematics Strategy Group at a series of the 
meetings held at the Annenberg Foundation Trust at Sunnylands in Rancho 
Mirage, California. The outcome of the meetings was a set of objectives 
involving large change to meet the need for independent reviews of materials. 
Since there was a lack of information needed to effect large change, structuring 
decisions made by Education First led to the choice of a planned change model 
to design EdReports.org. 


Implementing decisions involved using the planned change model to program 
procedures associated with the research, development, diffusion and adoption 
phases. The research phase involved EdReports staff conducting a listening 
tour to collect information about teachers’ use of materials, analysing rubrics, 
and observing reviewer training and reviewer evaluations of materials. The 
development phase involved the EdReports Board of Directors formulating a 
new solution for reviewing materials, drafting a business plan for EdReports.org, 
developing rubrics, a materials review process, a process for selecting 
reviewers and a training program for reviewers, and integrating these 
components into an operating system. The diffusion phase involved creating 
widespread awareness of EdReports program within the education community 
by publishing reports of reviews of materials and releasing news articles, 
posting blogs and education voices. The adoption phase involved collaborating 
with Orange County Department of Education in California to train advisors and 
teachers in a few districts to use EdReports reviews to select materials. Trial of 
the program in these districts led to a partnership with Pivot Learning to design 
the California Curriculum Collaborative, thereby fitting the characteristics of the 
program for widespread installation across California. The final activity of 
institutionalising EdReports program is represented by various projects 
commissioned by state education agencies in Massachusetts, Mississippi, 
Nebraska, New Mexico, Rhode Island, Tennessee, Texas and Wisconsin. 


Recycling decisions, precipitated by release of an open letter by the National 
Council of Teachers of Mathematics and the National Council of Supervisors of 
Mathematics raising concerns about the gateway procedure, led EdReports.org 
to refine the review tool, methodology and reporting protocol. 


2 


Conclusion 


The results of this study showed that the findings of research studies, 
recommendations set out in policy papers and widespread adoption of 
EdReports program by states and districts have implications for educational 
practice. Renewed interest by researchers into understanding the effects of 
decision-making at state and local levels for selecting high-quality, standards- 
aligned materials and the release of policy papers providing policymakers with 
recommendations to link high-quality, standards-aligned materials with 
professional learning are reflected in a key foundation funding grants. In 
January 2019, the Bill and Melinda Gates Foundation released a request for 
proposals to fund six to ten partnerships for teams of curriculum developers, 
professional learning providers and local education agencies to support 
implementation of high-quality materials for core subjects. 
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